NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization
نویسندگان
چکیده
0 and p. At any given time, a randomly selected agent is activated and performs computation to optimize its local objective. Such distributed computation model has been popular in large-scale machine learning and signal processing (6). Such model is also closely related to the (centralized) stochastic finite-sum optimization problem (14; 9; 13; 21; 1; 22), in which each time the iterate is updated based on the gradient information of a random ⇤Department of Industrial & Manufacturing Systems Engineering and Department of Electrical & Computer Engineering, Iowa State University, Ames, IA, {dhaji,mingyi}@iastate.edu †Department of Computer Science, Johns Hopkins University, [email protected] ‡Department of Operations Research, Princeton University,[email protected]
منابع مشابه
Decomposing Linearly Constrained Nonconvex Problems by a Proximal Primal Dual Approach: Algorithms, Convergence, and Applications
In this paper, we propose a new decomposition approach named the proximal primal dual algorithm (Prox-PDA) for smooth nonconvex linearly constrained optimization problems. The proposed approach is primal-dual based, where the primal step minimizes certain approximation of the augmented Lagrangian of the problem, and the dual step performs an approximate dual ascent. The approximation used in th...
متن کاملProx-PDA: The Proximal Primal-Dual Algorithm for Fast Distributed Nonconvex Optimization and Learning Over Networks
In this paper we consider nonconvex optimization and learning over a network of distributed nodes. We develop a Proximal Primal-Dual Algorithm (Prox-PDA), which enables the network nodes to distributedly and collectively compute the set of first-order stationary solutions in a global sublinear manner [with a rate of O(1/r), where r is the iteration counter]. To the best of our knowledge, this i...
متن کاملGradient Primal-Dual Algorithm Converges to Second-Order Stationary Solutions for Nonconvex Distributed Optimization
In this work, we study two first-order primal-dual based algorithms, the Gradient Primal-Dual Algorithm (GPDA) and the Gradient Alternating Direction Method of Multipliers (GADMM), for solving a class of linearly constrained non-convex optimization problems. We show that with random initialization of the primal and dual variables, both algorithms are able to compute second-order stationary solu...
متن کاملPerturbed Proximal Primal Dual Algorithm for Nonconvex Nonsmooth Optimization
In this paper we propose a perturbed proximal primal dual algorithm (PProx-PDA) for an important class of optimization problems whose objective is the sum of smooth (possibly nonconvex) and convex (possibly nonsmooth) functions subject to a linear equality constraint. This family of problems has applications in a number of statistical and engineering applications, for example in high-dimensiona...
متن کاملConvergence Rate Analysis of Primal-Dual Splitting Schemes
Primal-dual splitting schemes are a class of powerful algorithms that solve complicated monotone inclusions and convex optimization problems that are built from many simpler pieces. They decompose problems that are built from sums, linear compositions, and infimal convolutions of simple functions so that each simple term is processed individually via proximal mappings, gradient mappings, and mu...
متن کامل